Knowledge beta improvements #2545
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
brandonskiser
requested changes
Aug 12, 2025
crates/semantic-search-client/src/client/async_implementation.rs
Outdated
Show resolved
Hide resolved
kkashilk
reviewed
Aug 13, 2025
kensave
force-pushed
the
knowledge_beta_improvements
branch
2 times, most recently
from
f32f98c to
b55cf79
Compare
… (TI-3, TI-5) - Add pattern filtering foundation with --include/--exclude CLI flags (TI-3) - Implement knowledge settings integration with database system (TI-5) - Add comprehensive pattern filter module with glob-style support - Enhance file processor with async pattern filtering capabilities - Add extensive test coverage for pattern filtering functionality - Update knowledge CLI with improved error handling and validation - Add settings support for chunk size, overlap, and file limits
kensave
force-pushed
the
knowledge_beta_improvements
branch
from
b55cf79 to
2e1d733
Compare
brandonskiser
approved these changes
Aug 13, 2025
dingfeli
approved these changes
Aug 13, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pattern Filtering System (TI-3)
• CLI Integration: Added --include and --exclude flags to /knowledge add command
• Pattern Filter Module: New pattern_filter.rs with comprehensive glob-style pattern matching
• Async File Processing: Enhanced file_processor.rs with async pattern filtering capabilities
Minor fixes (TI-2)
• Rename from contexts from knowledge bases: Changed the semantic on how we refer to the knowledge entries.
• **"Are you sure?" : Changed confirmation message
• *Increased amount of extensions supported: Added more text extensions to list of supported files.
Move to right path for knowledge (TI-1)
• Automatic migration and new path for knowledge bases: Now they will live in ./aws/amazonq/knowledge_bases
Database Settings Integration (TI-5)
• Settings Schema: Added new settings for KnowledgeChunkSize, KnowledgeChunkOverlap, KnowledgeMaxFiles, KnowledgeDefaultIncludePatterns, KnowledgeDefaultExcludePatterns
• Helper Method: Added get_int_or() utility method for cleaner database settings access
• Configuration Integration: Knowledge store now reads chunk size, overlap, and file limits from database settings
🔧 API Improvements
Clean POJO-Based API
• New Request Structure: Replaced messy method overloads with AddContextRequest POJO
• Removed Deprecated Methods: Eliminated add_context_from_path() and add_context_from_path_with_patterns()
• Single Clean Method: New add_context(AddContextRequest) method handles all scenarios
• Better Naming: Renamed IndexingParams to AddContextRequest for external-facing API
Code Organization
• Extracted Migration Logic: Moved legacy knowledge base migration code to separate method
• Removed Verbose Patterns: Eliminated repetitive get_int().map_or() database access patterns
• Cleaner Error Handling: Improved error messages and validation throughout
🎨 User Experience Improvements
CLI Enhancements
• Better Styling: Changed add operation from green OperationResult::Success to neutral OperationResult::Info
• Improved Messages: Enhanced feedback messages with pattern filtering information
• Consistent Behavior: Both add and update operations now use consistent styling
Pattern Filtering UX
• Flexible Options: Support for include-only, exclude-only, or combined filtering
• Clear Feedback: Operations show which patterns are applied
• Database Defaults: Automatic loading of default patterns from database settings
API Consistency
• Method Signatures: Standardized all context operations to use structured request objects
• Error Handling: Consistent error types and messages across all operations
• Type Safety: Proper type definitions for all public API structures
📚 Documentation Updates
README Overhaul
• Async API Examples: Updated all code examples to use new async AddContextRequest API
• Pattern Filtering Guide: Comprehensive documentation with glob syntax and examples
• Feature Updates: Added pattern filtering and database settings to features list
• Removed Outdated Info: Eliminated references to deprecated sync API methods